In 2024 presidential polling, we’re seeing substantial mode effects in key subgroups. Phone-only polls are fairly uncommon, but several such polls — like the NYT/Siena poll — play an outsize role in shaping news coverage and media narratives of the campaign. While the toplines differ little between phone and online polls, there are stark differences in estimates of Trump support among key subgroups. Phone-only polls, in particular, have reported implausibly strong support for Trump among traditionally Democratic subgroups — including young people and Latinos.
We should expect a lot of variance in the crosstabs of any given poll. But given that these are relatively consistent patterns that appear when aggregating across polls, the findings reported here should caution against the reliability of phone polls.
This analysis is based on cross-tabs from national polls conducted since the start of 2024. I group polls into two mode categories: phone-only and online. Phone-only polls are those that are conducted completely over the phone. I categorize polls as conducted “Online” if any data collection happened via an online mode, even if the poll also included a telephone component. There are 6 phone-only polls and 31 online polls. All phone-only polls were conducted using probability-based samples. While we have 37 national polls in total, not all polls report crosstabs, so the sample sizes for each analysis differ.
Finally, while I restrict my analysis to polls conducted in 2024, I’ll briefly note that similar issues were apparent in the NYT/Siena battleground poll released in late October as well.
Let’s begin with age. In 2020 there was a sharp age gradient. According to Catalist’s estimates, Trump won just 32% of the two-party vote among 18-29 year-olds, 42% among 30-44 year-olds, 52% among 45-64 year-olds, and 52% among voters 65+. With that background, we have a strong prior that younger voters should be dramatically more supportive of Biden than older voters.
Yet, that expected age gradient only shows up in online polls. In phone polls since the start of 2024, there is virtually no age gradient at all. On average, phone polls put Trump’s 2-party support among under 20 at just over 50%. In contrast, the estimate in online polls is, on average, around 40-45%. Among people in their 50s, phone polls put Trump’s support in the 50-55% range — just a bit higher than young voters. Compare that to online polls, which put Trump’s support in the 55-60% range among this age bracket.
In short, phone polls are reaching much more pro-Trump young people than we would expect, based on 2020 results and comparison to online polls. Given the longstanding challenges with phone surveys, these results suggest that there may be substantial differential nonresponse among hard-to-reach young people in phone surveys.
Moving on to race, we see very large differences in race cross-tabs between phone and online polls. Specifically, phone-only polls have higher estimates of Trump support among Black and Latino voters. In the average online poll, two-party support for Trump among Latinos is about 41%, compared to 53% in the average phone-only poll. The Online estimate is right in line with 2020 results: Kuriwaki et al. (2023) estimate that Trump’s 2020 two-party vote share among Latinos was 38%.
There is a smaller but notable mode effect among Black respondents. In the average online poll, Trump’s two-party share among Black voters is 22%, compared to 25% in phone-only polls. Both of these estimates are much higher than the Trump vote share among Black voters in 2020 — estimated at 10% according to Kuriwaki et al.
Once again, these results suggest that a developing narrative about Trump’s growing support among racial and ethnic minorities is highly contingent on survey methods.
Finally, we see slightly higher education polarization in phone polls compared to online polls. In online polls, there is a 14-point education gap (43% to 57%); in phone-only polls, the education gap is a bit higher, at 16 points (42% to 58%).